Recovery of Information from Commercial Web Portals

ABSTRACT

Novel tools and techniques for automated recovery of information from commercial web portals, including commercial web portals requiring credentials for access. In some instances images are captured and pushed to external processes for improving system performance. In some instances access to automated software agent remote control modules is balanced across a system comprising a plurality of processors hosting the automated software agent remote control modules. Some instances provide provisioning of credentials, in particular indicating credentials available for an unlimited or a select limited number of users and monitoring credential use of those credentials usable by a select number of users. Some instances provide cache management optimizes retrieval of data by external processes and ensures reliability of such data to reduce unnecessary web portal inquiries.

RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.14/602,049, filed Jan. 21, 2015, which is a Continuation of U.S. patentapplication Ser. No. 13/356,461, filed Jan. 23, 2012, each entitled“Recovery of Information from Commercial Web Portals,” each of which ishereby incorporated by reference.

COPYRIGHT STATEMENT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

FIELD

The present disclosure relates, in general, to recovery of informationfrom commercial web portals, and more particularly, to automatedrecovery of patient specific information from insurance company webportals by emulation of an interactive user.

BACKGROUND

Companies often need to elicit from other companies. For example,companies providing insured services (those who are paid in whole or inpart by a party under a contract obligation, other than the partyreceiving the service) need certain information upon, or prior toacceptance of work, or delivery of services. In an insurance context,examples of the information needed may include:

Validity of coverage

-   -   Dates and what procedures are covered    -   Limitations    -   Co-Payments    -   Maximums    -   Deductibles    -   Thresholds    -   Reimbursement rates

Contractual obligations of provider and payer

In some instances it may also be necessary to receive a specificauthorization prior to rendering a service.

Obtaining information of this nature is currently done either through acomputer to computer transaction using an electronic data interchange(“EID”) format or via telephone call from the service provider to thepayer (i.e. insurer).

In an effort to streamline the flow of information and to minimizecosts, payers have begun making information of the type described aboveavailable as interactive transactions over the internet to their websiteor web portal. As a result, time consuming and therefore expensive phonecalls tying up the personnel time of the payer and the provider can beminimized as the provider can interactively query the payer's website toobtain this information. The information on the payer web portals isoften more comprehensive than that available over EID transactions.

As an alternative to providers obtaining the information directlythemselves, some third-party data sources will aggregate and providesuch information on a fee-per-transaction basis.

Unfortunately, each of these common methods of obtaining information hasrecognized problems. EID data is often missing or incomplete. Telephonecalls are very expensive due to personnel costs. Interactively accessinga payer's website is expensive, due to personnel costs associated withboth extracting information from the websites and learning to navigatevarious payer websites, each of which may be unique in the presentationof information. Finally, third-party data sources are expensive and mayraise concerns regarding the accuracy of data provided.

Accordingly, there is a need for tools and techniques that automate theprocess of querying payer web portals to acquire necessary informationand authorizations while minimizing or eliminating expenditure ofpersonnel time.

Automating the process of extracting information from payer web portalsin an automated manner would ideally enable processing an inboundrequest from another program (or “external process”) that supplies theinformation recovery system with search and navigation parameters, payerinformation, provider information and patient demographics to performthe desired search against the payer's web portal. However, developmentof such an automated system has proven very difficult for at least thefollowing reasons:

-   -   Each payer's website is different in content and format.    -   Each payer's website is different in navigation rules.    -   Navigation to the needed data can be very complex.    -   Privacy rules require that users accessing a web portal be        authorized to do so. In healthcare, the proof of authorization        is referred to as a “credential” and a valid credential must be        used to access the web portal.    -   Each provider has a very small number of credentials available        for each payer.    -   There generally are too few credentials to really use or to        attach to a session for extended periods.    -   Each credential may generally only be used serially. Any attempt        to use a credential in more than one concurrent process will        often be read as a security breach by the target web portal and        the credential may be invalidated for future use.    -   A payer web portal may have a limited number of concurrent        processes that may access it. This may be per source, per        provider or in total for the site.

Hence, there is a need for more robust tools and techniques foraccessing payer website data in an automated fashion.

BRIEF SUMMARY

A set of embodiments provides a software tool referred to herein as an“automated software agent” for interactively “impersonating” a humanuser conducting an Internet inquiry of an internet portal. Aspects ofthe automated software agent can include a set of “rules” and“configuration data,” so that the Automated Software Agent can accessand navigate various web portals to find desired information and/or readand parse resultant data streams. Aspects can further include parsingalgorithms and program processes so that the automated software agentcan recognize necessary information sets. Aspects of some embodimentsinclude methods for “pushing” image information to external processes orprograms. Aspects of other embodiments may further include controllingaccess to Automated Software Agent remote control modules across anetwork of cloud processers to make the remote control modules availablefor use to access discrete web portals in an efficient and organizedmanner.

Aspects of some embodiments include structured provisioning ofcredentials for efficiently accessing web portals, as required byexternal processes. In such embodiments, a pool of credentials might bemaintained along with the credential properties to allow for allocationof credentials as necessary to access web portals for obtaining desiredsets of patient-specific information. Aspects of embodiments can furtherinclude monitoring credential use and availability to optimizecredential availability and minimize conflicting demands for credentialsby the Automated Software Agent or Automated Software Agent remotecontrols. Aspects of some embodiments utilize cache management tominimize the need for time consuming queries of web portals wheredesired information, though time sensitive, can be retrieved from thecache with a high degree of reliability.

The tools provided by various embodiments include, without limitation,methods, systems, and/or software products. Merely by way of example, amethod might comprise one or more procedures, any or all of which areexecuted by a computer system. Correspondingly, an embodiment mightprovide a computer system configured with instructions to perform one ormore procedures in accordance with methods provided by various otherembodiments. Similarly, a computer program might comprise a set ofinstructions that are executable by a computer system (and/or aprocessor therein) to perform such operations. In many cases, suchsoftware programs are encoded on physical, tangible and/ornon-transitory computer readable media (such as, to name but a fewexamples, optical media, magnetic media, and/or the like).

Specific embodiments of the disclosure include a method of pushingobtained images of a portal page containing information sets recoveredfrom an insurance company web portal to an external process (e.g.program). The method comprises accessing an insurance company web portalwith an automated software agent. An image of a portal page containing aset of patient-specific information is obtained with the automatedsoftware agent. A unique filename for the image is created and the imageis stored in a database. The automated software agent invokes at leastone external process and pushes the unique filename to the at least oneexternal process. In some cases, invoking and pushing the uniquefilename to the external process may comprise identifying an operatingsystem on which the automated software agent executes and selecting amethod of invoking the external process based at least in part on theidentification of the operating system. Embodiments may further compriseassociating a processing status with filename, the processing statusindicating at least one of whether the external process is in theprocess of executing the filename or has already executed the filename.In such embodiments the external process may execute a filename only ifit is not in the process of executing or has not already executed thefilename. Embodiments may further include invoking more than oneexternal process with the automated software agent and passing theunique filename to each invoked external process.

Further specific embodiments of the disclosure may include a method ofload balancing automated software agent remote control modulereservations across an automated software agent grid comprising morethan one cloud processor hosting automated software agent remote controlmodules. One such method comprises providing a global automated softwareagent remote control module pool database comprising a list of webbrowsers supported by the automated software agent. Also provided is anautomated software agent remote control module provisioner databasecomprising a list of processors hosting automated software agent remotecontrol modules. Further provided is a remote control databasecomprising a list of automated software agent remote control modulesassociated with each processor. The computer system might receive a userrequest for an automated software agent remote control module andmatches the user request to one of the list of web browsers. Thecomputer system might query the software agent remote control moduleprovisioner database to identify a processor of the list of processorshaving the fewest active automated software agent remote control modulessupporting the matched web browsers. An automated software agent remotecontrol module hosted by the identified processor is selected. Areference to the selected automated software agent remote control moduleis returned in response to the user request.

Embodiments may further include determining if a selected automatedsoftware agent remote control module is functioning properly. If theselected automated software agent remote control module is notfunctioning properly, the selected automated software agent remotecontrol module is marked. Such embodiments may further include selectinganother automated software agent remote control module while skippingany marked automated software agent remote control modules. Suchembodiments may further include setting a timer when marking theselected automated software agent remote control module and when thetimer expires, deleting the marking from the selected automated softwareagent remote control module.

A further specific embodiment of the disclosure is a method ofprovisioning credentials to an automated software agent for accessing aninsurance company web portal for the purpose of obtainingpatient-specific information for a patient of interest. The method maycomprise providing a credentials database for storing credentialsuseable by the automated software agent for accessing an insurancecompany web portal. Various properties of the credential are associatedwith the credential. Properties may include, by way of example, eachrole the credential can enable on the insurance company web portal. Eachcredential and its associated properties are stored in the credentialsdatabase. The automated software agent identifies a credential for usein accessing a select insurance company web portal. The automatedsoftware agent queries the database to determine if the identifiedcredential can enable a select role or a select group of roles and todetermine if the identified credential is available for use. Thecomputer system provides the identified credential to the automatedsoftware agent for its use in accessing the select insurance company webportal only if the identified credential can enable the select role orgroup of roles and the identified credential is available for use andafter setting an in-use window having an in-use expiration time andassociating the in-use expiration time with the identified credentialand storing the in-use expiration time in the credentials database.

This embodiment may include, if the user cannot access the insurancecompany web portal using the identified credential, associating with theidentified credential a lockout window having a lockout expiration time.The computer system stores the lockout expiration time in thecredentials database and prevents use of the credential untiltermination of the lockout expiration time.

This embodiment may further include the computer system associating witheach credential any expiration date set by the insurance company webportal and updating the credential as necessary to allow access to theinsurance company web portal.

This embodiment may further include the computer system determining if acredential to be stored in the credentials database may be usedsimultaneously by unlimited multiple users to access an insurancecompany web portal. In this circumstance the computer system associateswith the credential an unlimited use role. For each credential with anunlimited use role, the setting of an in-use expiration time isprevented when the credential is returned to a user as an identifiedcredential. In this manner in-use expiration time is always in the pastwhich results in the credential always being available to a user.

This embodiment may also include the computer system determining if acredential stored in the credentials database may be used simultaneouslyby a select number of multiple users to access an insurance company webportal. In such an embodiment, the computer system associates a counterwith the credential to count the number of users simultaneously usingthe credential and does not return the credential to the automatedsoftware agent if the number of users simultaneously using thecredential counted by the counter is equal to the select number ofusers.

Another aspect of this embodiment may include a plurality of instancesof the automated software agent, each of the plurality of the instancesof the automated software agent having associated therewith anidentifier that uniquely identifies that instance of the automatedsoftware agent within the computer system. This aspect would furtherinclude maintaining a list of identifiers of currently running instancesof the automated software agent, with the list associating with eachcurrently running instance of the automated software agent one or morecredentials currently used by that currently-running instance of theautomated software agent, a status of that instance of the automatedsoftware agent and a set of one or more dispatch parameterscorresponding to that instance of the automated software agent.“Dispatch parameters” might include, by way of example, a timestamp(start, end), method invoked, credential id, payer URL, and patient IDand other such information. Such an aspect may further includeidentifying from the list of identifiers, a conflict between two or moreinstances of the currently running automated software agent andresolving the conflict by terminating one or more of the currentlyrunning instances of the automated software agent.

Aspects of this embodiment may further include determining from the listof identifiers that a currently running instance of the automatedsoftware agent has failed and terminating the failed currently runninginstance of the automated software agent.

A further aspect may include analyzing how the instances of theautomated software agent accesses resources for the purpose ofeliminating future conflicts between instances of the automated softwareagent acquiring a credential.

Yet another specific embodiment of the disclosure is a method ofutilizing a computer cache to optimize obtaining time variableinformation from a web portal using an automated software agent.Embodiments comprise accessing, with an automated software agent, a webportal containing time variable information. The automated softwareagent obtains a set of elements of time variable information. The set oftime variable information is date stamped and stored in a computer cacheas a reference set of elements of time variable data. The automatedsoftware agent accesses the web portal containing the time variableinformation and obtains a later set of elements of the time variableinformation. The later set of elements of the time variable informationis date stamped. The automated software agent compares the reference setof elements of time variable information and the later set of elementsof the time variable information to identify any changes between commonelements of the reference set of elements of time variable informationand the later set of elements of the time variable information. Elementsof the reference set of time variable information are replaced withcommon elements of the later set of the time variable information in thecomputer cache if there are differences between the common elements ofthe reference set of time variable information and the later set of thetime variable information. Each element of the reference set of elementsof time variable information and the corresponding element of the laterset of elements of the time variable information are analyzed as afunction of the difference between date stamps to establish criteria fordetermining whether the automated software agent can rely on thereference set of data in the cache in a future query to obtain anelement of time variable information or whether the element of timevariable information must be accessed from the web portal. The criteriaare associated and stored with each element of the reference set of timevariable data in a database.

Aspects of this embodiment may further include the step of analyzingeach element of the reference set of elements including storinginformation about comparisons in a database and evaluating informationin this database to optimize criteria of data type and age fordetermining whether the automated software agent can rely on thereference set of data in the cache in future queries. The optimizedcriteria are associated and sorted with each element of the referenceset of time variable data in a database.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of particularembodiments may be realized by reference to the remaining portions ofthe specification and the drawings, in which like reference numerals areused to refer to similar components. In some instances, a sub-label isassociated with a reference numeral to denote one of multiple similarcomponents. When reference is made to a reference numeral withoutspecification to an existing sub-label, it is intended to refer to allsuch multiple similar components.

FIG. 1 is a functional block diagram illustrating a system for recoveryof information from commercial web portals in accordance with variousembodiments.

FIG. 2 is a process flow diagram illustrating a general process forobtaining information from a web portal, in accordance with variousembodiments.

FIG. 3 is a process flow diagram illustrating a method of pushingobtained images of portal page information sets recovered from a webportal to an external process, in accordance with various embodiments.

FIG. 4 is a process flow diagram of a method of load balancing automatedsoftware agent remote control module reservations across a system forrecovery of information from commercial web portals, in accordance withvarious embodiments.

FIG. 5 is a process flow diagram of a method of provisioning credentialsto an automated software agent for accessing insurance company webportals, in accordance with various embodiments.

FIG. 6 is a process flow diagram of a method of identifying andresolving conflicts between instances of an automated software agent, inaccordance with various embodiments.

FIG. 7 is a process flow diagram of a method of utilizing a computercache to optimize obtaining time variable information from a commercialweb portal using an automated software agent, in accordance with variousembodiments.

FIG. 8 is a generalized schematic diagram illustrating a computersystem, in accordance with various embodiments.

FIG. 9 is a block diagram illustrating a networked system of computers,which can be used in accordance with various embodiments.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

While various aspects and features of certain embodiments have beensummarized above, the following detailed description illustrates a fewexemplary embodiments in further detail to enable one of skill in theart to practice such embodiments. The described examples are providedfor illustrative purposes and are not intended to limit the scope of theinvention.

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the described embodiments. It will be apparent to oneskilled in the art, however, that other embodiments of the present maybe practiced without some of these specific details. In other instances,certain structures and devices are shown in block diagram form. Severalembodiments are described herein, and while various features areascribed to different embodiments, it should be appreciated that thefeatures described with respect to one embodiment may be incorporatedwith other embodiments as well. By the same token, however, no singlefeature or features of any described embodiment should be consideredessential to every embodiment of the invention, as other embodiments ofthe invention may omit such features.

Unless otherwise indicated, all numbers used herein to expressquantities, dimensions, and so forth used should be understood as beingmodified in all instances by the term “about.” In this application, theuse of the singular includes the plural unless specifically statedotherwise, and use of the terms “and” and “or” means “and/or” unlessotherwise indicated. Moreover, the use of the term “including,” as wellas other forms, such as “includes” and “included,” should be considerednon-exclusive. Also, terms such as “element” or “component” encompassboth elements and components comprising one unit and elements andcomponents that comprise more than one unit, unless specifically statedotherwise.

FIG. 1 is a functional block diagram of a system 110 for recovery ofinformation from commercial web portals. The system 110 comprises anautomated software agent 115 which may one or more programs, services,daemons, scripts and/or other executable code hosted on a single ormultiple processors. A number of users 120 interface with the automatedsoftware agent 115. The users 120 direct requests for information to becollected by the automated software agent 115 from various commercialweb portals. The automated software agent 115 is associated with a cache125 for transient storage of information and a database 130 forpersistent storage of information such as credentials used for accessingweb portals, statistics related to management of the cache 125 andinformation related to management of remote controls operativelyassociated with the automated software agent 115.

The automated software agent 115 is further in communication with anumber of processors illustrated as 135-1, 135-2, 135-n, which might beimplemented as different computers, different processors or cores withina single computer, or using any other appropriate hardware architecture.In other cases, the functionality attributed to the multiple processors135 can be performed by a single processor. A number of automatedsoftware agent remote control instances indicated as RC₁-RC_(n) areassociated with each processor. The remote control instances RC₁-RC_(n)are functional for accessing web portals for accessing desiredinformation as will be described in greater detail below. As used hereinthe “automated software agent” includes the remote control instances.Reference to an “instance” of the automated software agent is intendedto include instances of the automated software agent as a whole or an“instance” of a software remote control agent, unless it is clear in thecontext that “instance” is intended to be limited to only the automatedsoftware agent as a whole.

In one embodiment the system 110 including the automated software agent115 comprises a set of computer programs that take a request from a user120 (also referred to as an external process or program) and processthat request interactively as if it were a human operator againstvarious web portals 140 to gather desired information in an automatedmanner. One example of a tool that can be implemented as part of as theautomated software agent platform is an open source product calledSelenium™, which acts as a driver between the users 120 and the payer'sweb portals 140. Selenium™ is an open source software testing tool thatallows user to drive different browsers from a Java-based application totest their own web portals in an automated manner. Selenium™ is made upof three components, Selenium™ Core, Selenium™ Remote Control andSelenium™ Grid.

Selenium™ Core consists of a Java scrip API that runs inside the browserto control the browser.

Selenium™ Remote Control acts as a driver for the Selenium™ Core. Itcomes in two parts. The first is a Java based proxy application thattakes HTTP, posts commands and forwards them to Selenium™ Core runninginside the appropriate browser. It launches different browsers and loadsthe Java script into them. Second, Selenium™ Remote Control comes withdrivers for multiple languages including the Java one.

Selenium™ Grid is a Java based applications that listens for HTTPcommands on a single port and passes those commands on to specificremote control instances. Selenium™ Grid allows Selenium™ RemoteControls to be run in a scaled production environment.

Critical to scaled and reliable application of the system 110, iseffective management of the remote controls. This is accomplished byeffectively deploying multiple grid configurations in the event a gridbecomes completely inactive or a grid is lost due to a hardware failure.

In some aspects, the automated software agent 115 comprises and accessesa number of remote control instances, each of which are associated withvarious processors, which could, for example, be part of a cloudconfiguration. The automated software agent 115 includes an automatedsoftware agent 115 grid program which enables the deployment of thevarious remote controls across multiple virtual and physical processors.In one embodiment, as discussed above, the automated software agent 115may be Selenium™.

One application of the system 110 is in the field of healthcare wherethe users 120 comprise various service providers seeking to access theweb portals of payers or insurers for the purpose of determininginformation related to the coverage of an insured person seekingservices from the provider. While in some instances the discussionherein will relate to this specific application of the system 110,unless entirely clear from the context, the system 110 is not intendedto be limited to this specific application.

FIGS. 2-7 illustrate various methods that can be used in associationwith the system 110 for recovery of information from commercial webportals. While the methods of FIGS. 2-7 are illustrated, for ease ofdescription, as different methods, it should be appreciated that thevarious techniques and procedures of these methods can be combined inany suitable fashion, and that, in some embodiments, the methodsdepicted by FIGS. 2-7 can be considered interoperable and/or as portionsof a single method. Similarly, while the techniques and procedures aredepicted and/or described in a certain order for purposes ofillustration, it should be appreciated that certain procedures may bereordered and/or omitted within the scope of various embodiments.Moreover, while the methods illustrated by FIGS. 2-7 can be implementedby (and, in some cases, are described with respect to) the system 110 ofFIG. 1 (or components thereof), these methods may also be implementedusing any suitable hardware implementation. Similarly, while the system110 of FIG. 1 (and/or components thereof) can operate according to themethods illustrated by FIGS. 2-7 (e.g., by executing instructionsembodied on a computer readable medium), the system 110 can also operateaccording to other modes of operation and/or perform other suitableprocedures.

For instance, FIG. 2 is a process flow diagram illustrating a generalprocess for obtaining information from a web portal using the system110. In one embodiment the method 210 is implemented using the automatedsoftware agent platform described above, which can be a Java ApplicationProgram Interface (“API”) that opens a URL to a commercial web portaland interacts dynamically through implementation of the method 210.

Each implementation and instance of the method 210 might implement astandard interface. The interface takes a plain old Java object (POJO)that contains information necessary to extract desired information fromthe commercial web portal. For example, in an embodiment where aprovider is accessing an insurer's website, such information may includeinformation about the payer, information about the provider, informationsought from the web portal and the provider's credentials to use whenlogging in to the payer's portal. The method 210, in some cases, returnsdata from the web portal (which can include an image of the web portal,as described below) or error messages, as appropriate.

At block 215, the method 210 comprises receiving and/or validating therequest for information (e.g., patient information) from a particularweb portal. Validation comprises verifying that the request includessufficient information to do at least one patient search. Embodimentsmay validate other input information based on known restrictions of thepayer (e.g., member ID length must be a select number of characters). Ifthe request passes validation, at block 220 an instance of an automatedsoftware agent remote control is accessed (and/or started as necessary).The automated software agent remote control is in turn used to navigateto a web portal login screen at block 225 and log in to the web portalat block 230. Once the remote control has successfully logged in, atblock 235, it navigates the web portal to perform a patient search.Different web portals may allow searching for patient information inmore than one way. Embodiments may, for example, allow searching bymember ID alone or they may require last name or date of birthinformation. To accommodate the demands of various web portals, themethod 210 utilizes a waterfall search in various embodiments to performthe patient search. Once the desired patient information is identified,the requested information is captured (block 240); such informationwhich can include, without limitation, available information about thepatient's plan and demographics, such as a patient's member ID, name,date of birth, relationship code, gender and plan description. In somecases, the captured information includes an image of the web portal fromwhich the information was captured. At 245 the remote control logs outand becomes available for use by another instance.

In order to provide substantiation that the response returned from anexecution of the automated software agent 115 is accurate, an image istaken from any portal page from which information is extracted. In anaspect, an image can be any representation of a web portal or page thatincludes sufficient information to allow validation and/orsubstantiation that the captured information was captured from theappropriate web portal and/or pertains to the appropriate patient. Thus,an image can include, inter alia, a screen capture of a portal web page,a timestamp, a URL, a digital signature and/or certificate, and/or otherverification information, and/or the like. These images may be useful inthe event some other user or external process has a need for theinformation recovered, or if there is ever a question about the validityof the information. To make the image available, a filename is createdfor the image and the image is moved to a substantiation storeaccessible by external processes. Typically, such external processeswould poll the substantiation database for image data. However, pollingfor this data is inefficient because it increases network traffic anddisc input and output. Thus, certain embodiments provide tools forproviding image data without requiring polling by external processes.

Merely by way of example, FIG. 3 is a process flow diagram of a method310 of “pushing” obtained images of a portal page to an externalprocess. At block 315 an image of a portal page is created (e.g., aspart of the method 210 described above). In some embodiments, a browseraccessing the portal page can take an image of an entire page. In otherinstances, for example when the browser is Internet Explorer, an imagecan only be taken of a portion of a page that is visible at the time theimage is captured. To work around this issue, embodiments includescrolling and capturing a plurality of screen images as necessary toobtain the full image of a portal page when the automated software agent115 is executed. Once the image is captured, a unique file name iscreated for the image at block 320. At block 325, the image is stored ina substantiation database accessible by various external processes thatcan be invoked.

In some cases, external processes (such as instances of the softwareagent 115) might execute on a variety of different platforms havingdifferent operating systems. In such cases, the procedures for invokingthe external process might vary between platforms and/or operatingsystems. Embodiments may further include, at block 330, identifying anoperating system of the automated software agent 115. At 335, dependingon the operating system of the automated software agent 115, embodimentsmay select a method to invoke various external processes that may have aneed for the stored image. For example, if the operating system isLinux, a Java Runtime Executor might execute a shell script (e.g., aBash script), Perl script, and/or the like. If the operating system isWindows, a Java Runtime Executor is used to execute a DOS batch file.Alternatively and/or additionally, a platform-specific compiled launcherprogram might be used to invoke an external process. In any event, atblock 340 the external process or processes are invoked (e.g., using theselected technique, depending on the platform). At block 345, thefilename is “pushed” to the external processes for execution by theexternal processes, for example, by passing the filename to theinvocation script, etc.

Certain embodiments could also include (at block 350) associating aprocess status with a filename. For example, the process status couldindicate at least one of whether the external process is in the processof executing the filename or has already executed the filename. Incertain embodiments, if the external process is executing the filename(e.g., processing the image referenced by the filename or taking someother specified action with respect to the file) or has already executedthe filename the external process will not execute the pushed filename.

FIG. 4 is a functional block diagram of a method 410 of load balancingautomated software remote control reservations across a system forrecovery of information from commercial web portals. The methodcomprises, at block 415, providing a number of databases. As usedherein, the term “providing,” with respect to a database, can includeone or more of several operations, including without limitation creatingthe database, maintaining (e.g., compacting, etc.) the database, makingthe database available for queries, storing the database (e.g., on acomputer readable medium), and/or the like. Essentially, providing adatabase can include any action that makes the database available foruse by various embodiments. A number of different databases may beprovided in accordance with different embodiments. For example, a remotecontrol pool database might include a list of web browsers supported bythe automated software agent. The web browsers could include, but arenot necessarily limited to, Microsoft Internet Explorer, Mozilla Firefoxand Google Chrome. Some embodiments might include an automated softwareagent remote control provisioner database comprising a list ofprocessors hosting automated software agent remote controls. Otherembodiments might include a remote control database comprising a list ofautomated software agent remote control modules associated with eachprocessor. (It should be noted, of course, that, while these databasesare described separately for illustrative purposes, these databases canbe provided as one or more tables within a single database; in othercases, there might be additional databases, and/or data can be dividedbetween one or more databases in a fashion different than that describedherein. Thus, the number and arrangement of the databases isimplementation-specific, and the examples provided herein should not beconsidered limiting.)

At block 420, a user request for an automated software agent remotecontrol is received by the system 110. Such a request can be receivedfrom a variety of entities (each of which can be considered a “user” forpurposes of the method 410), including without limitation, a human user,an external process (running on the system 110 or another computer),and/or the like. At block 425, the user request is matched to acompatible web browser. For example, a particular request might pertainto a web portal that requires the use of Internet Explorer. In such acase, that request might be matched with Internet Explorer as thecompatible web browser. At block 430, the provisioner database isqueried to identify a processor from a list of processers having thefewest active automated software agent remote control modules supportingthe matched web browser. At block 435 an automated software agent remotecontrol module is selected from the identified processor. In an aspect,the selected module executes on the identified processor.

A number of techniques can be used to prioritize the selection of aremote control module. For example, in some cases, a round-robinallocation technique can be used. In other cases, existing load on eachprocessor might be taken into account when selecting a remote controlmodule. Other considerations are possible as well. In one aspect, theselection operation might include skipping (i.e., disregarding forselection purposes) any instances of the automated software agent remotecontrol module that are marked as not functioning properly. Thus, atblock 440, the method 410 comprises determining whether the selectedautomated software agent remote control is functioning properly. Atblock, 445 a functioning, selected automated software agent remotecontrol module is returned in response to the request. However, if it isdetermined that the selected automated software agent remote control isnot functioning properly, the automated software agent remote controlthat is not functioning properly is marked (block 450), and the method410 returns to step 420 to repeat the selection process. In some cases,when an agent is marked as non-functioning, the method 410 includessetting a timer (block 455) to specify a duration for which the moduleshould be marked non-functional, in order to enable later selection ofthe remote control module (e.g., in case the module returns tofunctional status in the interim). At 460, after expiration of thetimer, the marking is deleted.

The method 410 can be repeated from step 425 until a remote control thatfunctions properly is selected in response to the request. If anautomated software agent remote control module that functions properlycannot be identified, an error is returned in response to the request.

In an aspect, the method 410 of load balancing automated software agentremote control reservations can spread the use of remote controls overany number of virtual or actual processes associated with the grid. Thiscan prevent aggregation of remote control instances on a single machine,thereby enhancing performance of the system as a whole. Furthermore, themethod 410 allows for searching for a functioning remote controlinstance across all available processors in a grid system, so that allpossible remote control instances can be investigated before an error isreturned to the requester. Further, the method 410 recognizes thatremote control instances can become unresponsive for reasons other thana complete remote control failure. For example, browser hangs, machinefailure and network connectivity issues all can present transientnon-functionality conditions. The use of the marker and timer allows thesystem to determine if the remote control failure is temporary so thatthe remote control can reenter use if the failure to function is notattributable to a complete remote control failure.

Another aspect of certain embodiments provides control over credentialuse. As used herein, a “credential” is a valid authorized useridentifier and password that is used with the permission of both therequesting user and the owner of the servicing web portal. For anautomated software agent remote control to access many commercial webportals, it must use a valid credential. In general, credentials mayonly be used serially. An attempt to use a credential in more than oneconcurrent process will often be read as a security breach by the webportal and the credential may be invalidated for future use. For asystem 110 for recovery of information from commercial website portalsto work efficiently, effective management of access to credentials iscritical.

Hence, FIG. 5 illustrates a method 510, in accordance with one set ofembodiments, of provisioning credentials to an automated software agent(or an automated software agent remote control) for accessing acommercial web portal for the purpose of obtaining desired information.In one embodiment, the method may be used for accessing an insurancecompany web portal for the purpose of obtaining patient-specificinformation for a patient of interest, although various embodiments arenot limited to this particular use case. The method 510 manages the useof credentials across multiple applications (which can be running onmultiple physical or virtual processers) for a variety of purposes.Useful features of the method 510 include, without limitation, enablingaccess to a credentials database for instances of an automated softwareagent (or automated software agent remote control) to have access tocurrent and correct credential information. Another useful feature canbe to allow for credentials to be reserved from and returned to acredentials pool so that only one web session may have an individualcredential in use at any given time, if appropriate, and when finished,the credential is rendered available for use by another instance of theautomated software agent or automated software agent remote control. Afurther useful feature can enable establishment of priorities for use ofcredentials amongst competing users and assign credentials from the“pool” based on these priorities. Yet another useful feature canaccommodate situations where multiple users may access a credential atonce and if the number of users is limited, tracking the number of usersto prevent a violation of limitations on the number of credential users.The method 510 can also provide credentials with a high likelihood offunctioning properly.

The method 510 includes, at block 515, providing a credentials databasefor storing credentials. At block 520, the method 510 includesassociating properties and/or roles with each credential (e.g., bystoring in the database an association between a credential and theapplicable properties or roles, either as fields in a credentialrecords, relational links between credential records and propertyrecords, or any other appropriate technique, which can depend on thedatabase architecture, which, as noted above, is discretionary).Properties may include, for example, the identity of a commercial webportal, user name and password associated with the credential, whetherthe credential is checked in and thus available or checked out andtherefore unavailable, a time when a checked out credential will bechecked in and different roles a credential can enable. “Roles” may beapplication specific or user specific. For example, returning to theexample of the web portal being a payer portal and the user beingmedical service provider, a role could exist for claims status andeligibility and benefits applications. Roles could also exist fordeveloper and production users. For example, if a credential can only beused by production users of eligibility and benefits applications, thetwo roles would be specified as properties in the credentials database.If it could also be used by production users in the claims statusapplication, a further claims status application would be added to thecredential. In various embodiments, a credential can be assigned manyroles which are then checked when a credential is investigated for use.Additional properties might include whether a particular credential maybe used simultaneously by multiple users. At block 525, the method 510includes determining whether a credential can be used simultaneously bymultiple users. If the credential may be used simultaneously by multipleusers, a multiple use role is associated with the credential propertiesfor such credentials (block 530). In some embodiments, the multiple userole might allow for unlimited concurrent use by multiple users. Inother embodiments, the multiple use role enable only a limited selectnumber of concurrent users. At block 535, the credential and associatedproperties are stored in the credentials database.

At block 540, the method 510 comprises querying the credentialsdatabase\for credentials to access a select web portal. In an exemplaryembodiment, the query might identify the web portal and propertiesrequired of the credential, such as roles the credential can enable.

At block 545, the system determines whether the identified credential isavailable for use. For example, if a credential is a single-use (orlimited-use) credential, an in-use window might be set (block 550) toindicate that the credential is currently in-use (and thereforeunavailable) with the in-use window including an in-use expiration time,which is then added to the credential properties in the credentialsdatabase. If the credential is marked as in-use, the system woulddetermine (at block 545) that the credential is not available. In thoseinstances where the credential has a multiple use role enablingunlimited concurrent users to use the credential, setting of an in-useexpiration time is prevented. In this manner, the in-use expiration timefor such an unlimited use credential would always be in the past andresult in the credential always being available to a user. In thoseinstances where a credential may be used simultaneously by a selectnumber of multiple users, a counter is associated with the credential tocount the number of users simultaneously using the credential. Themethod will not return a credential to the automated software agent ifthe number of users simultaneously using the credential counted by thecounter is equal to the select number of users.

At block 555 an identified credential is provided by the system to theautomated software agent, if available. At block 560, the methodcomprises determined if the credential can provide access to the webportal (e.g., whether the agent can successfully log on to the portalwith the credential). If not, a lockout window having a lockoutexpiration time is associated with the credential, and the lockoutexpiration time is added to the properties in the credentials database(block 565). The system prevents the use of the credential untiltermination of the lockout expiration time. After the expiration of thelockout time, the software agent (or another software agent) mightre-attempt to access the portal. This can prevent fatal lockoutconditions for transient failures (such as a case where the portalhappens to be generally unavailable at a certain time, due to user load,maintenance, etc., simultaneous use of the credential by the actualpatient, etc.)

Embodiments may also include associating with each credential anyexpiration date set by the web portal. Such embodiments may furtherinclude the system updating the credential as necessary to allow accessto the insurance company web portal.

Embodiments may also be provided to include functionality foridentifying instances of automated software agent remote control (or anautomated software agent) and identifying and resolving conflictsbetween instances of automated software agent remote controls, so as toprevent the instances of the automated software agent remote controlsfrom tying up resources (such as, for example a race condition betweentwo instances). One such method 610 is illustrated in FIG. 6. In theillustrated embodiment, a plurality of instances of the automatedsoftware agent remote controls are provided (block 615). At block 620,an identifier is associated with each instance of the automated softwareagent remote control. At block 625, a list of running instances ofautomated software agents is maintained (e.g., in a database), and atblock 630 credentials currently in use, current status and dispatchparameters are associated with each automated software agent remotecontrol instance. At block 635, conflicts between instances of automatedsoftware agent remote controls are identified and at block 640, suchconflicts are resolved at 640. Examples of “conflicts” include, but arenot limited to, competition for a credential or a duplication of acredential. The conflicts may be resolved, for example by terminating aninstance of one of the conflicting automated software agent remotecontrols. Another embodiment may include a step of determining from thecurrent status of an instance of an automated software agent remotecontrol that the remote control has failed. Such an embodiment wouldfurther include terminating the failed currently running instance of theautomated software agent remote control. The method may further includethe step of analyzing instances of how the automated software agentremote control accesses resources for the purpose of eliminating futureconflicts between instances of automated software agent remote controlsacquiring a credential.

Embodiments of the system 110 of FIG. 1 can also include tools andtechniques for optimizing a system cache 125. Optimizing cacheutilization can eliminate a resource consuming access of a web portalwhile still providing accurate time variable information.

A method 710 of utilizing a computer cache to optimize obtaining timevariable information from a commercial web portal using an automatedsoftware agent is illustrated in FIG. 7. The method 710 includesaccessing, with an automated software agent, a web portal containingtime variable information (block 715). The set of elements of timevariable information may include an image of a web portal page, otherinformation captured from the page, etc. At block 725, the set of timevariable information is date stamped, and at block 730 the set of timevariable information is stored as a reference set in the computer cache.At block 735, a later set of elements of the time variable informationis accessed with the automated software agent from the web portal. Thislater set of elements of time variable information is date stamped atblock 740. At block 745, the reference set and the later set of elementsof time variable information are compared to identify any changesbetween common elements of the reference set of elements of timevariable information and the later set of elements of time variableinformation. At block 750, the reference set is updated by replacing theelements of the reference set of time variable information with commonelements of the later set of time variable information in the computercache, if there are differences between the common elements of thereference set of time variable information and the later set of timevariable information. At block 755 each element of the reference set ofelements of time variable information and the corresponding elements ofthe later set of elements of time variable information are analyzed as afunction of the difference between the date stamps to establish criteriafor determining whether the automated software agent can rely on thereference set (or elements of the data comprising the reference set) inthe cache in a future query to obtain an element of time variableinformation or whether the element of time variable information must beaccessed from the web portal. For example, if a number of sequentialcaptures of the information indicates that the information is static fora period of several months before changing, a reference set of the datathat is younger than this length of might be assumed to be current,while a reference set of the data older than this length of time mightbe assumed to be out-of-date, in which case the data needs to berecaptured from the portal (and the reference data refreshed, asdescribed above). At block 760, these criteria are associated with andstored with each element of the reference set of time variableinformation in the database.

Embodiments may further include optimizing the criteria at block 765.The operation of optimizing can comprise storing information about thecomparisons in a database and evaluating information in this database tooptimize criteria of data type captured from the portal, properties ofthe request (for example, the nature of a procedure to be performed, thepayer, the provider, geographic information (e.g., state, county, SMSA),insured group (e.g., teachers, auto workers, teamsters), patientoccupation, date (e.g., time variable information may be known to varyat specific dates for example at the new year)) and age of the data inthe cache for determining whether the automated software agent can relyon the reference set of data in the cache in accordance with therequirements of the provider or payer in future queries. For example,certain providers may demand that approval of certain procedures requirea higher degree of reliability for the time variable information that isstored in the cache than other procedures, and thus for such proceduresan element of time variable information must be accessed from the webportal while for another procedure the time variable information couldbe retrieved from the cache. At block 770, the optimized criteria areassociated with and stored with each element of the reference set oftime variable information in the database.

Embodiments may further including adding any element of the later set oftime variable information not in the reference set of elements of timevariable information to the reference set of time variable information.

FIG. 8 provides a schematic illustration of one embodiment of a computersystem 800 that can perform the methods provided by various otherembodiments, as described herein, and/or can function as the users 120or processor s135-1 to 135-n described above. It should be noted thatFIG. 8 is meant only to provide a generalized illustration of variouscomponents, of which one or more (or none) of each may be utilized asappropriate. FIG. 8, therefore, broadly illustrates how individualsystem elements may be implemented in a relatively separated orrelatively more integrated manner.

The computer system 800 is shown comprising hardware elements that canbe electrically coupled via a bus 805 (or may otherwise be incommunication, as appropriate). The hardware elements may include one ormore processors 810, including without limitation one or moregeneral-purpose processors and/or one or more special-purpose processors(such as digital signal processing chips, graphics accelerationprocessors, and/or the like); one or more input devices 815, which caninclude without limitation a mouse, a keyboard and/or the like; and oneor more output devices 820, which can include without limitation adisplay device, a printer and/or the like.

The computer system 800 may further include (and/or be in communicationwith) one or more storage devices 825, which can comprise, withoutlimitation, local and/or network accessible storage, and/or can include,without limitation, a disk drive, a drive array, an optical storagedevice, solid-state storage device such as a random access memory(“RAM”) and/or a read-only memory (“ROM”), which can be programmable,flash-updateable and/or the like. Such storage devices may be configuredto implement any appropriate data stores, including without limitation,various file systems, database structures, and/or the like.

The computer system 800 might also include a communications subsystem830, which can include without limitation a modem, a network card(wireless or wired), an infra-red communication device, a wirelesscommunication device and/or chipset (such as a Bluetooth™ device, an802.11 device, a WiFi device, a WiMax device, a WWAN device, cellularcommunication facilities, etc.), and/or the like. The communicationssubsystem 830 may permit data to be exchanged with a network (such asthe network described below, to name one example), with other computersystems, and/or with any other devices described herein. In manyembodiments, the computer system 800 will further comprise a workingmemory 835, which can include a RAM or ROM device, as described above.

The computer system 800 also may comprise software elements, shown asbeing currently located within the working memory 835, including anoperating system 840, device drivers, executable libraries, and/or othercode, such as one or more application programs 845, which may comprisecomputer programs provided by various embodiments, and/or may bedesigned to implement methods, and/or configure systems, provided byother embodiments, as described herein. Merely by way of example, one ormore procedures described with respect to the method(s) discussed abovemight be implemented as code and/or instructions executable by acomputer (and/or a processor within a computer); in an aspect, then,such code and/or instructions can be used to configure and/or adapt ageneral purpose computer (or other device) to perform one or moreoperations in accordance with the described methods.

A set of these instructions and/or code might be encoded and/or storedon a non-transitory computer readable storage medium, such as thestorage device(s) 825 described above. In some cases, the storage mediummight be incorporated within a computer system, such as the system 800.In other embodiments, the storage medium might be separate from acomputer system (i.e., a removable medium, such as a compact disc,etc.), and/or provided in an installation package, such that the storagemedium can be used to program, configure and/or adapt a general purposecomputer with the instructions/code stored thereon. These instructionsmight take the form of executable code, which is executable by thecomputer system 800 and/or might take the form of source and/orinstallable code, which, upon compilation and/or installation on thecomputer system 800 (e.g., using any of a variety of generally availablecompilers, installation programs, compression/decompression utilities,etc.) then takes the form of executable code.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware (such as programmable logic controllers,field-programmable gate arrays, application-specific integratedcircuits, and/or the like) might also be used, and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

As mentioned above, in one aspect, some embodiments may employ acomputer system (such as the computer system 800) to perform methods inaccordance with various embodiments of the invention. According to a setof embodiments, some or all of the procedures of such methods areperformed by the computer system 800 in response to processor 810executing one or more sequences of one or more instructions (which mightbe incorporated into the operating system 840 and/or other code, such asan application program 845) contained in the working memory 835. Suchinstructions may be read into the working memory 835 from anothercomputer readable medium, such as one or more of the storage device(s)825. Merely by way of example, execution of the sequences ofinstructions contained in the working memory 835 might cause theprocessor(s) 810 to perform one or more procedures of the methodsdescribed herein.

The terms “machine readable medium” and “computer readable medium,” asused herein, refer to any medium that participates in providing datathat causes a machine to operation in a specific fashion. In anembodiment implemented using the computer system 800, various computerreadable media might be involved in providing instructions/code toprocessor(s) 810 for execution and/or might be used to store and/orcarry such instructions/code (e.g., as signals). In manyimplementations, a computer readable medium is a non-transitory,physical and/or tangible storage medium. Such a medium may take manyforms, including but not limited to, non-volatile media, volatile media,and transmission media. Non-volatile media includes, for example,optical and/or magnetic disks, such as the storage device(s) 825.Volatile media includes, without limitation, dynamic memory, such as theworking memory 835. Transmission media includes, without limitation,coaxial cables, copper wire and fiber optics, including the wires thatcomprise the bus 805, as well as the various components of thecommunication subsystem 830 (and/or the media by which thecommunications subsystem 830 provides communication with other devices).Hence, transmission media can also take the form of waves (includingwithout limitation radio, acoustic and/or light waves, such as thosegenerated during radio-wave and infra-red data communications).

Common forms of physical and/or tangible computer readable mediainclude, for example, a floppy disk, a flexible disk, a hard disk,magnetic tape, or any other magnetic medium, a CD-ROM, any other opticalmedium, punch cards, paper tape, any other physical medium with patternsof holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chipor cartridge, a carrier wave as described hereinafter, or any othermedium from which a computer can read instructions and/or code.

Various forms of computer readable media may be involved in carrying oneor more sequences of one or more instructions to the processor(s) 810for execution. Merely by way of example, the instructions may initiallybe carried on a magnetic disk and/or optical disc of a remote computer.A remote computer might load the instructions into its dynamic memoryand send the instructions as signals over a transmission medium to bereceived and/or executed by the computer system 800. These signals,which might be in the form of electromagnetic signals, acoustic signals,optical signals and/or the like, are all examples of carrier waves onwhich instructions can be encoded, in accordance with variousembodiments of the invention.

The communications subsystem 830 (and/or components thereof) generallywill receive the signals, and the bus 805 then might carry the signals(and/or the data, instructions, etc. carried by the signals) to theworking memory 835, from which the processor(s) 805 retrieves andexecutes the instructions. The instructions received by the workingmemory 835 may optionally be stored on a storage device 825 eitherbefore or after execution by the processor(s) 810.

As noted above, a set of embodiments comprises systems for recovery ofinformation from commercial web portals. FIG. 9 illustrates a schematicdiagram of a system 900 that can be used in accordance with one set ofembodiments. The system 900 can include one or more user computers 905.A user computer 905 can be a general purpose personal computer(including, merely by way of example, desktop computers, laptopcomputers, handheld computers, and the like, running any appropriateoperating system, several of which are available from vendors such asApple, Microsoft Corp., and the like) and/or a workstation computerrunning any of a variety of commercially-available UNIX™ or UNIX-likeoperating systems. A user computer 905 can also have any of a variety ofapplications, including one or more applications configured to performmethods provided by various embodiments (as described above, forexample), as well as one or more office applications, database clientand/or server applications, and/or web browser applications.Alternatively, a user computer 905 can be any other electronic device,such as a thin-client computer, Internet-enabled mobile telephone,and/or personal digital assistant, capable of communicating via anetwork (e.g., the network 910 described below) and/or of displaying andnavigating web pages or other types of electronic documents. Althoughthe exemplary system 900 is shown with three user computers 905, anynumber of user computers can be supported.

Certain embodiments operate in a networked environment, which caninclude a network 910. The network 910 can be any type of networkfamiliar to those skilled in the art that can support datacommunications using any of a variety of commercially-available (and/orfree or proprietary) protocols, including without limitation TCP/IP,SNA™, IPX™, AppleTalk™, and the like. Merely by way of example, thenetwork 910 can include a local area network (“LAN”), including withoutlimitation a fiber network, an Ethernet network, a Token-Ring™ networkand/or the like; a wide-area network; a wireless wide area network(“WWAN”); a virtual network, such as a virtual private network (“VPN”);the Internet; an intranet; an extranet; a public switched telephonenetwork (“PSTN”); an infra-red network; a wireless network, includingwithout limitation a network operating under any of the IEEE 802.11suite of protocols, the Bluetooth™ protocol known in the art, and/or anyother wireless protocol; and/or any combination of these and/or othernetworks.

Embodiments can also include one or more server computers 915. Each ofthe server computers 915 may be configured with an operating system,including without limitation any of those discussed above, as well asany commercially (or freely) available server operating systems. Each ofthe servers 915 may also be running one or more applications, which canbe configured to provide services to one or more clients 905 and/orother servers 915. Merely by way of example, in some cases, one or moreservers 915 might be configured to operate a core system of an operatedsoftware agent, while other servers 915 (or clients) can be configuredto operate remote control modules, as described above. In someinstances, a plurality of servers 915 (i.e., processors) might beconfigured to operate in a grid or “cloud” configuration, in whichcomputing resources (memory, processors, etc.) are shared as a pool, onwhich different applications (e.g., software agents) can draw.

In some cases, one of the servers 915 may be a web server, which can beused, merely by way of example, to process requests for web pages orother electronic documents from user computers 905. The web server canalso run a variety of server applications, including HTTP servers, FTPservers, CGI servers, database servers, Java servers, and the like. Insome embodiments of the invention, the web server may be configured toserve web pages that can be operated within a web browser on one or moreof the user computers 905 to perform methods of the invention.

The server computers 915, in some embodiments, might include one or moreapplication servers, which can be configured with one or moreapplications accessible by a client running on one or more of the clientcomputers 905 and/or other servers 915. Merely by way of example, theserver(s) 915 can be one or more general purpose computers capable ofexecuting programs or scripts in response to the user computers 905and/or other servers 915, including without limitation web applications(which might, in some cases, be configured to perform methods providedby various embodiments). Merely by way of example, a web application canbe implemented as one or more scripts or programs written in anysuitable programming language, such as Java™, C, C#™ or C++, and/or anyscripting language, such as Perl, Python, or TCL, as well ascombinations of any programming and/or scripting languages. Theapplication server(s) can also include database servers, includingwithout limitation those commercially available from Oracle™,Microsoft™, Sybase™, IBM™ and the like, which can process requests fromclients (including, depending on the configuration, dedicated databaseclients, API clients, web browsers, etc.) running on a user computer 905and/or another server 915. In some embodiments, an application servercan create web pages dynamically for displaying the information inaccordance with various embodiments. Data provided by an applicationserver may be formatted as one or more web pages (comprising HTML,JavaScript, etc., for example) and/or may be forwarded to a usercomputer 905 via a web server (as described above, for example).Similarly, a web server might receive web page requests and/or inputdata from a user computer 905 and/or forward the web page requestsand/or input data to an application server. In some cases a web servermay be integrated with an application server.

In accordance with further embodiments, one or more servers 915 canfunction as a file server and/or can include one or more of the files(e.g., application code, data files, etc.) necessary to implementvarious disclosed methods, incorporated by an application running on auser computer 905 and/or another server 915. Alternatively, as thoseskilled in the art will appreciate, a file server can include allnecessary files, allowing such an application to be invoked remotely bya user computer 905 and/or server 915.

It should be noted that the functions described with respect to variousservers herein (e.g., application server, database server, web server,file server, etc.) can be performed by a single server and/or aplurality of specialized servers, depending on implementation-specificneeds and parameters.

In certain embodiments, the system can include one or more databases920, such as those described above. The location of the database(s) 920is discretionary: merely by way of example, a database 920 a mightreside on a storage medium local to (and/or resident in) a server 915 a(and/or a user computer 905). Alternatively, a database 920 b can beremote from any or all of the computers 905, 915, so long as it can bein communication (e.g., via the network 910) with one or more of these.In a particular set of embodiments, a database 920 can reside in astorage-area network (“SAN”) familiar to those skilled in the art.(Likewise, any necessary files for performing the functions attributedto the computers 905, 915 can be stored locally on the respectivecomputer and/or remotely, as appropriate.) In one set of embodiments,the database 920 can be a relational database, such as an Oracledatabase, that is adapted to store, update, and retrieve data inresponse to SQL-formatted commands. The database might be controlledand/or maintained by a database server, as described above, for example.

While certain features and aspects have been described with respect toexemplary embodiments, one skilled in the art will recognize thatnumerous modifications are possible. For example, the methods andprocesses described herein may be implemented using hardware components,software components, and/or any combination thereof. Further, whilevarious methods and processes described herein may be described withrespect to particular structural and/or functional components for easeof description, methods provided by various embodiments are not limitedto any particular structural and/or functional architecture but insteadcan be implemented on any suitable hardware, firmware and/or softwareconfiguration. Similarly, while certain functionality is ascribed tocertain system components, unless the context dictates otherwise, thisfunctionality can be distributed among various other system componentsin accordance with the several embodiments.

Moreover, while the procedures of the methods and processes describedherein are described in a particular order for ease of description,unless the context dictates otherwise, various procedures may bereordered, added, and/or omitted in accordance with various embodiments.Moreover, the procedures described with respect to one method or processmay be incorporated within other described methods or processes;likewise, system components described according to a particularstructural architecture and/or with respect to one system may beorganized in alternative structural architectures and/or incorporatedwithin other described systems. Hence, while various embodiments aredescribed with—or without—certain features for ease of description andto illustrate exemplary aspects of those embodiments, the variouscomponents and/or features described herein with respect to a particularembodiment can be substituted, added and/or subtracted from among otherdescribed embodiments, unless the context dictates otherwise.Consequently, although several exemplary embodiments are describedabove, it will be appreciated that the invention is intended to coverall modifications and equivalents within the scope of the followingclaims.

What is claimed is:
 1. A method of utilizing a computer cache tooptimize obtaining time variable information from a commercial webportal using an automated software agent, the method comprising:accessing, with an automated software agent, a web portal containingtime variable information; obtaining, with the automated software agent,a set of elements of time variable information; date stamping the set oftime variable information; storing the set of time variable informationwith the date stamp in a computer cache as a reference set of elementsof time variable data; accessing, with the automated software agent, theweb portal containing the time variable information; obtaining, with theautomated software agent, a later set of elements of the time variableinformation; date stamping the later set of elements of the timevariable information; comparing, with the automated software agent, thereference set of elements of time variable information and the later setof elements of the time variable information to identify any changesbetween common elements of the reference set of elements of timevariable information and the later set of elements of the time variableinformation; replacing the elements of the reference set of timevariable information with common elements of the later set of the timevariable information in the computer cache if there are differencesbetween the common elements of the reference set of time variableinformation and the later set of the time variable information;analyzing each element of the reference set of elements of time variableinformation and the corresponding element of the later set of elementsof the time variable information as a function of the difference betweendate stamps to establish criteria for determining whether the automatedsoftware agent can rely on the reference set of data in the cache in afuture query to obtain an element of time variable information orwhether the element of time variable information must be accessed fromthe web portal; and associating and storing the criteria with eachelement of the reference set of time variable data in a database.
 2. Themethod of claim 1 wherein the step of analyzing each element of thereference set of elements further comprises storing information aboutcomparisons in a database and evaluating information in this database tooptimize criteria of data type and age for determining whether theautomated software agent can rely on the reference set of data in thecache in future queries; and associating and storing the optimizedcriteria with each element of the reference set of time variable data ina database.
 3. The method of claim 1 further comprising adding anyelement of the later set of time variable information not in thereference set of elements of time variable information to the referenceset of time variable information.
 4. A computer system comprising: oneor more processors; a database operatively associated with at least oneof the one or more processors; a cache operatively associated with atleast one of the one or more processors; an automated software agentoperatively associated with at least one of the set of processors; a setof instructions executable by at least one of the one or moreprocessors, to cause the at least one of the one or more processors toperform one or more operations, the set of instructions comprising:instructions for accessing, with an automated software agent, a webportal containing time variable information; instructions for obtaining,with the automated software agent, a set of elements of time variableinformation; instructions for date stamping the set of time variableinformation; instructions for storing the set of time variableinformation with the date stamp in the computer cache as a reference setof elements of time variable data; instructions for accessing, with theautomated software agent, the web portal containing the time variableinformation; instructions for obtaining, with the automated softwareagent, a later set of elements of the time variable information;instructions for date stamping the later set of elements of the timevariable information; instructions for comparing, with the automatedsoftware agent, the reference set of elements of time variableinformation and the later set of elements of the time variableinformation to identify any changes between common elements of thereference set of elements of time variable information and the later setof elements of the time variable information; instructions for replacingthe elements of the reference set of time variable information withcommon elements of the later set of the time variable information in thecomputer cache if there are differences between the common elements ofthe reference set of time dependent information and the later set of thetime variable information; instructions for analyzing each element ofthe reference set of elements of time variable information and thecorresponding element of the later set of elements of the time variableinformation as a function of the difference between date stamps toestablish criteria for determining whether the automated software agentcan rely on the reference set of data in the cache in a future query toobtain an element of time variable information or whether the element oftime variable information must be accessed from the web portal; andinstructions for associating and storing the criteria with each elementof the reference set of time variable data in a database.
 5. Anapparatus comprising: a computer readable medium having stored thereon aset of instructions executable by one or more computers to perform oneor more operations, the set of instructions comprising: instructions foraccessing, with an automated software agent, a web portal containingtime variable information; instructions for obtaining, with theautomated software agent, a set of elements of time variableinformation; instructions for date stamping the set of time variableinformation; instructions for storing the set of time variableinformation with the date stamp in a computer cache as a reference setof elements of time variable data; instructions for accessing, with theautomated software agent, the web portal containing the time variableinformation; instructions for obtaining, with the automated softwareagent, a later set of elements of the time variable information;instructions for date stamping the later set of elements of the timevariable information; instructions for comparing, with the automatedsoftware agent, the reference set of elements of time variableinformation and the later set of elements of the time variableinformation to identify any changes between common elements of thereference set of elements of time variable information and the later setof elements of the time variable information; instructions for replacingthe elements of the reference set of time variable information withcommon elements of the later set of the time variable information in thecomputer cache if there are differences between the common elements ofthe reference set of time dependent information and the later set of thetime variable information; instructions for analyzing each element ofthe reference set of elements of time variable information and thecorresponding element of the later set of elements of the time variableinformation as a function of the difference between date stamps toestablish criteria for determining whether the automated software agentcan rely on the reference set of data in the cache in a future query toobtain an element of time variable information or whether the element oftime variable information must be accessed from the web portal; andinstructions for associating and storing the criteria with each elementof the reference set of time variable data in a database.